AprilTag with Python

In this tutorial, you will learn how to perform AprilTag detection with Python and the OpenCV library.

AprilTags are a type of fiducial marker. Fiducials, or more simply “markers,” are reference objects that are placed in the field of view of the camera when an image or video frame is captured.

The computer vision software running behind the scenes then takes the input image, detects the fiducial marker, and performs some operation based on the type of marker and where the marker is located in the input image.

AprilTags are a specific type of fiducial marker, consisting of a black square with a white foreground that has been generated in a particular pattern (as seen in the figure at the top of this tutorial).

The black border surrounding the marker makes it easier for computer vision and image processing algorithms to detect the AprilTags in a variety of scenarios, including variations in rotation, scale, lighting conditions, etc.

You can conceptually think of an AprilTag as similar to a QR code — a 2D binary pattern that can be detected using computer vision algorithms. However, an AprilTag only holds 4-12 bits of data, multiple orders of magnitude less than a QR code (a typical QR code can hold up to 3KB of data).

So, why bother using AprilTags at all? Why not simply use QR codes if AprilTags hold such little data?

The fact that AprilTags store less data is actually a feature and not a bug/limitation. To paraphrase the official AprilTag documentation, since AprilTag payloads are so small, they can be more easily detected, more robustly identified, and less difficult to detect at longer ranges.

Basically, if you want to store data in a 2D barcode, use QR codes. But if you need to use markers that can be more easily detected in your computer vision pipeline, use AprilTags.

Fiducial markers such as AprilTags are an integral part of many computer vision systems, including but not limited to:

One of the primary benefits of AprilTags is that they can be created using basic software and a printer. Just generate the AprilTag on your system, print it out, and include it in your image processing pipeline — Python libraries exist to automatically detect the AprilTags for you!

In the rest of this tutorial, I will show you how to detect AprilTags using Python and OpenCV.

To learn how to detect AprilTags with OpenCV and Python, just keep reading.

In the first part of this tutorial, we will discuss what AprilTags and fiducial markers are. We’ll then install apriltag, the Python package we’ll be using to detect AprilTags in input images.

Next, we’ll review our project directory structure and then implement our Python script used to detect and identify AprilTags.

We’ll wrap up the tutorial by reviewing our results, including a discussion on some of the limitations (and frustrations) associated with AprilTags specifically.

AprilTags are a type of fiducial marker. Fiducials are special markers we place in the view of the camera such that they are easily identifiable.

For example, all of the following tutorials used fiducial markers to measure either the size of an object in an image or the distance between specific objects:

Successfully implementing these projects was only possible because a marker/reference object was placed in view of the camera. Once I detected the object, I could derive the width and height of other objects because I already know the size of the reference object.

AprilTags are a special type of fiducial marker. These markers have the following properties:

Once detected in a computer vision pipeline, AprilTags can be used for:

A great example of using fiducials could be in a large fulfillment warehouse (i.e., Amazon) where you’re using autonomous forklifts.

You could place AprilTags on the floor to define “lanes” for the forklifts to drive on. Specific markers could be placed on large shelves such that the forklift knows which crate to pull down.

And markers could even be used for “emergency shutdowns” where if that “911” marker is detected, the forklift automatically stops, halts operations, and shuts down.

There are an incredible number of use cases for AprilTags and the closely related ArUco tags. I’ll be covering the basics of how to detect AprilTags in this tutorial. Future tutorials on the PyImageSearch blog will then build off this one and show you how to implement real-world applications using them.

In order to detect AprilTags in our images, we first need to install a Python package to facilitate AprilTag detection.

The library we’ll be using is apriltag, which, lucky for us, is pip-installable.

To start, make sure you follow my pip install opencv guide to install OpenCV on your system.

If you are using a Python virtual environment (which I recommend, since it is a Python best practice), make sure you use the workon command to access your Python environment and then install apriltag into that environment:

From there, validate that you can import both cv2 (your OpenCV bindings) and apriltag (your AprilTag detector library) into your Python shell:

Congrats on installing both OpenCV and AprilTag on your system!

All that said, are you:

Then join PyImageSearch Plus today! Gain access to PyImageSearch tutorial Jupyter Notebooks that run on Google Colab’s ecosystem right in your browser! No installation required.

And best of all, these notebooks will run on Windows, macOS, and Linux!

Before we implement our Python script to detect AprilTags in images, let’s first review our project directory structure:

Here you can see that we have a single Python file, detect_apriltag.py. As the name suggests, this script is used to detect AprilTags in input images.

We then have an images directory that contains two example images. These images each contain one or more AprilTags. We’ll use our detect_apriltag.py script to detect the AprilTags in each of these images.

With the apriltag Python package installed, we are now ready to implement AprilTag detection with OpenCV!

Open up the detect_apriltag.py file in your project directory structure, and let’s get started:

We start off on Lines 2-4 importing our required Python packages. We have:

From here, Lines 7-10 parse our command line arguments. We only need a single argument here, --image, the path to our input image containing the AprilTags we want to detect.

Next, let’s load our input image and preprocess it:

Line 14 loads our input image from disk using the supplied --image path. We then convert the image to grayscale, the only preprocessing step required for AprilTag detection.

Speaking of AprilTag detection, let’s go ahead and perform the detection step now:

In order to detect AprilTags in an image, we first need to specify options, and more specifically, the AprilTag family:

A family in AprilTags defines the set of tags the AprilTag detector will assume in the input image. The standard/default AprilTag family is “Tag36h11”; however, there are a total of six families in AprilTags:

You can read more about the AprilTag families on the official AprilTag website, but for the most part, you typically use “Tag36h11”.

Line 20 initializes our options with the default AprilTag family of tag36h11.

From there, we:

The final step here is to loop over the AprilTags and display the results:

We start looping over our AprilTag detections on Line 26.

Each AprilTag is specified by a set of corners. Lines 29-33 extract the four corners of the AprilTag square, while Lines 36-39 draw the AprilTag bounding box on the image.

We also compute the center (x, y)-coordinates of the AprilTag bounding box and then draw a circle representing the center of the AprilTag (Lines 42 and 43).

The last annotation we’ll perform is grabbing the detected tagFamily from the result object and then drawing it on the output image as well.

Finally, we wrap up our Python by displaying the results of our AprilTag detection.

Let’s put our Python AprilTag detector to the test! Make sure you use the “Downloads” section of this tutorial to download the source code and example image.

From there, open up a terminal, and execute the following command:

Despite the fact that the AprilTag has been rotated, we were still able to detect it in the input image, thereby demonstrating that AprilTags have a certain level of robustness that makes them easier to detect.

Let’s try another image, this one with multiple AprilTags:

Here we have a fleet of autonomous vehicles, each with an AprilTag placed on it. We are able to detect all AprilTags in the input image, except for the ones that are partially obscured by other robots (which makes sense — the entire AprilTag has to be in view for us to detect it; occlusion creates a big problem for many fiducial markers).

Be sure to use this code as a starting point for when you need to detect AprilTags in your own input images!

You may have noticed that I did not cover how to manually generate your own AprilTag images. That’s for two reasons:

All that said, I find generating AprilTags to be a pain in the ass. Instead, I prefer to use ArUco tags, which OpenCV can both detect and generate using it’s cv2.aruco submodule.

I’ll be showing you how to use the cv2.aruco module to detect both AprilTags and ArUco tags in a tutorial in late-2020/early-2021. Be sure to stay tuned for that tutorial!

In this tutorial, we used example images of AprilTags from other websites. I would like to take a second and credit the official AprilTag website as well as Bernd Pfrommer from the TagSLAM documentation for the examples of AprilTags.

I strongly believe that if you had the right teacher you could master computer vision and deep learning.

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s not the case.

All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.

Inside PyImageSearch University you'll find:

Click here to join PyImageSearch University

In this tutorial, you learned about AprilTags, a set of fiducial markers that are often used for robotics, calibration, and 3D computer vision projects.

We use AprilTags (as well as the closely related ArUco tags) in these situations because they tend to be very easy to detect in real time. Libraries exist to detect AprilTags and ArUco tags in nearly any programming language used to perform computer vision, including Python, Java, C++, etc.

In our case, we used the april-tag Python package. This package is pip-installable and allows us to pass in images loaded by OpenCV, making it quite effective and efficient in many Python-based computer vision pipelines.

Later this year/in early 2021, I’ll be showing you real-world projects of using AprilTags and ArUco tags, but I wanted to introduce them now so you have a chance to familiarize yourself with them.

To download the source code to this post (and be notified when future tutorials are published here on PyImageSearch), simply enter your email address in the form below!

Enter your email address below to get a .zip of the code and a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!